1

Current Status of Python

Python is amazing. It is amazing because it has managed to stay relevant and keep growing for so many years.

For a very long time, one of the most important virtues of Python was interoperability. Thanks to that feature it didn't matter what operating system you or your customers were using. If the Python interpreter was available for a given operating system, your software written in Python would work there. And, most importantly, your software would always work the same way regardless of that operating system. However, this is now a common trait. Many modern programming languages provide similar interoperability capabilities. Also, with the advent of cloud computing, web-based applications, and reliable virtualization software, it isn't that important to have a programming language that works in many operating systems.

What seems to be increasingly more important for programmers nowadays is productivity. With a constant search for innovation, it is often important to build something that you can test in the field with real users and then iterate quickly from that point until you build a really valuable product. And Python allows programmers to iterate really fast. The Official Python Package Index is a tremendous library of software libraries and frameworks that can be easily reused in your software. It greatly reduces the amount of time and effort required to get your job done. This vast availability of community libraries together with clear and concise syntax that favors readability greatly limits the time and effort that has to be spent on creating and maintaining the software. That makes Python really shine in the area of programming productivity.

Python stayed relevant for so long because it was constantly evolving. And it still keeps evolving. That's why in this chapter we will take a brief look at the current status of Python and learn how to stay up to date with changes happening in the Python ecosystem and community.

In this chapter, we will cover the following topics:

• Where are we now and where are we going?
• What to do with Python 2
• Keeping up to date

Let's start by considering the history and development of Python, and where we're currently at.

Where are we now and where are we going?

Python isn't a young language. Its history starts somewhere in the late 1980s and the official 1.0 release happened in year 1994. We could discuss the whole timeline of major Python releases mentioned here, but we really only need to be concerned about a few dates:

• October 16, 2000: official release of Python 2.0
• December 3, 2008: official release of Python 3.0
• November 9, 2011: announcement of Python 2.8 release un-schedule
• January 1, 2020: official sunsetting of Python 2

So, at the time of writing, Python 3 is almost half as old as Python itself. It has also been active for longer than Python 2 was when it comes to active development of new language features.

Although Python 3 has been around for quite some time, its adoption was so slow that the initial end of life for Python 2 was postponed by 5 years. It was mostly due to a lot of backward compatibility issues that didn't always allow an easy and straightforward transition. A big part of Python's greatness comes from the vast number of freely available libraries. So, it was even harder to make the Python 3 transition if your software critically relied on a third-party Python package that wasn't compatible with Python 3 yet.

Fortunately, Python 2 is finally behind us and the Python community can finally breathe a sigh of relief. Many developers of open-source Python libraries stopped providing compatibility with Python 2 already a long time ago. Also, the official sunsetting of Python 2 provided an incentive to strategically prioritize transitions to Python 3 in corporate environments, where there is usually limited desire to do so. That incentive can be attributed mainly to the fact that there is absolutely no chance of security patches for Python 2 being delivered anymore.

What to do with Python 2

We know that Python 2 is no longer officially supported by language developers. Due to the lack of security patches, it should be considered unsafe. But is it dead yet?

Probably not. Even relatively popular open-source libraries can lose the interest of their authors and contributors over time. It also happens that some better alternatives appear, and there's simply no point in developing the original library anymore. Such libraries are often left unmaintained, so no one could update them for compatibility with Python 3.

One of the important reasons why Python 3 couldn't take off for a long time was the sluggish release processes of many Linux distributions. Distribution maintainers often aren't inclined toward the fast adoption of new language versions, especially if these versions break backward compatibility and require patches of other software. Many Python programmers are Linux users, and if they don't have access to the latest version of the language through the system package repository they are less likely to upgrade it on their own.

But the place where Python 2 will definitely linger for a few more years is corporate environments. When money is at stake, it is usually hard to convince the stakeholders that something that's already been done needs to be revisited just for the sake of being up to date with upstream changes, especially if the software works.

Python 2 code probably won't be that common for the core components of actively developed software, but it can still be found quite often in utility scripts, internal tools, or services that haven't seen active development for a long time.

If you're in a situation where you still need to maintain bits of software written in Python 2 you would be better off migrating to Python 3 soon. This often can't be done overnight, and sometimes you'll have to convince others first before you're ready to proceed.

If you're in such a situation, consider the following strategy:

1. Identify what you need to migrate and why: Depending on the situation, you'll be dealing with various pieces of code that have different uses. Not everything may need migration after all. If code isn't actually used by anyone, there is no value in keeping it updated.
2. Identify what is holding you back: Sometimes you'll be working with code that has dependencies that cannot be easily migrated. This will make the process a bit harder, so you'll have to know this in advance in order to create a good migration plan.
3. Convince the stakeholders: If you're developing an open-source library, you'll have to convince fellow contributors that will help you make this migration; if you're working at a software company, you'll have to convince the stakeholders who are paying for the job to be done that this is more important than, for instance, working on a new feature.

Usually the hardest thing to do is to get buy-in from stakeholders, especially if you're writing code professionally and need to find a way to squeeze such a project into the day-to-day development of new features. You'll need to be well prepared before raising such an issue. That's why the first two steps are essential. They allow you to estimate the amount of work required to make the change and construct a convincing reason for doing so. After all, the best way to convince someone is by presenting the list of benefits of doing a migration. The following are common positive reasons for doing a migration:

• Ability to use newer and better libraries: After the official sunsetting of Python 2 there is a very low chance that new (and possibly better) libraries will be compatible with Python 2.
• Reduced cost of ownership: If the team is using the latest version of Python in other projects/components, it will be cheaper to converge multiple projects to a single version as this will reduce the overall complexity.
• Simpler maintenance and operations: Different runtime environments and operating systems will gradually lose default built-in support for Python 2. Using a newer version of Python will limit the operational costs required to deploy software.
• Easier new hire onboarding: Sticking to a single version of Python makes it easier to onboard new team members as it will be easier for them to work with the whole codebase from the very beginning

Another tool for convincing stakeholders is explaining the risks related to not migrating to a newer version of Python:

• Big security risk: After the official end of life, there's no guarantee of any official security fixes. This risk is, of course, only speculative until real vulnerabilities are discovered. Still, by using Python 2 you're already limiting your ability to accept updates to third-party libraries and many open-source projects that dropped support for Python 2 a few years ago.
• High security-related cost: Although it is possible to fork open-source code and do security fixes on your own, the cost of doing this continuously is usually much bigger than the overall cost of migrating to a newer version of the language.
• Troubles in hiring new people: This is problematic regardless of target programmer seniority. Younger developers may not be as familiar with Python 2 as they are with Python 3. This will make their onboarding harder and also increase the risk of making rookie mistakes with potentially disastrous outcomes. Senior engineers on the other hand may not want to work in an environment that relies on outdated technology.

Regardless of your communication strategy, you'll have to create a reasonable tactical plan that carefully balances the promise of positive outcomes, risks, and the ability to reach team goals at a sustainable pace. For instance, investing in reducing future maintenance costs usually pays for itself only after a longer period of time. That's why it makes sense to also spread such investment over time. Known and exploitable security vulnerabilities, on the other hand, should always be prioritized.

Keeping up to date

Technology evolves constantly. People are constantly looking for tools that will allow them to solve their problems more easily than before. Every few months, either a completely new language pops up almost out of nowhere, or one of the well-established languages introduces a completely new syntax feature. This also happens to Python. We will briefly consider a few of the most important recent additions to the Python language in Chapter 3, New Things in Python.

New languages or language features drive the creation of novel libraries and frameworks. These in turn open the way for new programming paradigms and new design patterns. The success of one such pattern or paradigm in one language ecosystem often encourages programmers to adopt them in other languages. That's how new ideas spread from language to language.

We see that happening to Python too. We will be able to see how seemingly different programming languages share similar features or concepts in Chapter 4, Python in Comparison with Other Languages.

This process of language evolution is constant, inevitable, and seems to be only speeding up. Starting from Python 3.9 (released October 5, 2020), new major releases of Python will be happening annually. These releases more often than not lead to explosions of new libraries and frameworks that are trying to test and explore new tools. It's great for the Python community because it gives it constant fuel to innovate. But it can be overwhelming for anyone that wants to keep an eye on what's happening in Python. And being up to date is extremely important for every Python professional.

In the following few sections, we will discuss a few ways in which you can source information on what is happening in Python. It will allow you to better anticipate changes happening to the language and the community around it. This will keep you up to date with the latest best practices, and new tools that are worth investing in.

PEP documents

The Python community has a well-established way of dealing with changes. While speculative Python language ideas are mostly discussed on a specific mailing list (python-ideas@python.org), nothing major ever gets changed without the existence of a new document, called a Python Enhancement Proposal (PEP).

A PEP is a formalized document that describes, in detail, the proposal of change to be made in Python. It is also the starting point for the community discussion. A few of the topics covered in this chapter were in fact already extensively described in their dedicated PEP documents:

• PEP 373—Python 2.7 Release Schedule
• PEP 404—Python 2.8 Un-release Schedule
• PEP 602—Annual Release Cycle for Python

The whole purpose, format, and workflow of these documents is also standardized in the form of a standalone PEP—the PEP 1 document.

PEP documentation is very important for Python, and, depending on the topic, it serves different purposes:

• Informing: They summarize the information needed by core Python developers, and notify about Python release schedules
• Standardizing: They provide code style, documentation, or other guidelines
• Designing: They describe the proposed features

A list of all proposed PEPs is available in a living (by which we mean continuously updated) PEP 0 document. It is a great source of information for those who are wondering what direction Python is heading in but do not have time to track every discussion on Python mailing lists. It shows which ideas have been accepted, which ideas have been implemented, and also which ideas are still under consideration.

http://www.python.org/dev/peps/pep-XXXX

The index of all officially discussed proposals serves an additional purpose. Very often, people ask questions like the following:

• Why does feature A work that way?
• Why does Python not have feature B?

In most such cases, the answer is already available in the specific PEP where the feature was discussed. Instead of having the same conversation about adding some language feature over and over again, people can be redirected to a specific PEP. And there are a lot of PEPs describing Python language features that have been proposed but not accepted.

When it comes to being up to date, the most important section of the PEP index is definitely open PEPs, which describes ideas that are still under active consideration. For instance, the following are selected interesting PEPs that were still open at the time of writing:

• PEP 603—Adding a frozenmap type to collections
• PEP 634—Structural Pattern Matching: Specification
• PEP 638—Syntactic Macros
• PEP 640—Unused variable syntax

These proposals vary from relatively small extensions of the existing standard library (such as PEP 603) to completely new and complex syntax features (such as PEP 638). If you wonder what the Python community is considering including in future Python releases, then open PEPs are the best source of such information.

Active communities

The non-profit organization behind Python is Python Software Foundation (PSF). It holds intellectual property rights for Python and manages its licensing. A brief excerpt from its mission statement is as follows:

The mission of the Python Software Foundation is to promote, protect, and advance the Python programming language, and to support and facilitate the growth of a diverse and international community of Python programmers.

Support for the community of Python programmers is important to the PSF mission because it is, after all, the community that drives the development of Python. The community does that by extending the language transparently and openly (as explained in the PEP documents section), but also through expanding and maintaining a rich ecosystem of third-party packages and frameworks. So, one of the best ways to know what's happening in Python is to get in touch with its community.

Like any other programming language, there are a lot of independent online communities dedicated to Python. They are usually focused on specific frameworks or software development areas, like web development, data science, machine learning, and so on.

One could assume that there is at least a single place online where all important discussions about the core of the language and its interpreter happen. Unfortunately, it's not that simple.

Due to many reasons, some of which are historic, the landscape of official Python mailing lists and discussion boards can be very confusing. There are numerous official and semi-official mailing lists and discussion boards where Python developers hang out. It is especially confusing when it comes to mailing lists because official mailing lists are spread over two different mailing list managers and thus have two separate sets of list archives. Those archives are:

• Mailman 2: This is an older and smaller archive available at https://mail.python.org/mailman/listinfo. Historically all python.org mailing list archives could be accessed through the Mailman 2 archive but most of them have been migrated to the Mailman 3 archive. Anyway, there are still some mailing lists with active discussions that are managed through Mailman 2.
• Mailman 3: This is a younger archive available at https://mail.python.org/archives. It is currently a primary archive of python.org mailing lists and is the home for more active mailing lists. It has a more modern and convenient interface, but it doesn't include mailing lists that have not been migrated from Mailman 2 yet.

When it comes to actual mailing lists, there's plenty of them, but sadly the majority of them aren't active anymore. Some mailing lists are dedicated to specific projects (like scikit-image@python.org) or focus on specific areas of interest of their members (like code-quality@python.org). Besides lists with very specific themes, there are few general mailing lists that should be very interesting for every Python programmer. These are:

• python-ideas@python.org: This is a staple when it comes to Python mailing lists. It is a place for discussing a wide variety of ideas regarding Python. Most PEPs start as a speculative idea discussion on this mailing list. It is a great place for discussing potential "what ifs" and finding out what features people would like to see in the near future.
• python-dev@python.org: This is a mailing list specifically for the purpose of core Python development (mainly CPython interpreter). It is also a place where first drafts of new PEPs are discussed before being officially announced through other channels. It isn't a place where you should be asking for general Python help, but it is an essential resource if you would like to try your hand at fixing known bugs in the CPython interpreter or the Python standard library.
• python-announce-list@python.org: This is a mailing list for various announcements. You can find announcements of conferences and meetups here, as well as notifications about new releases of your favorite packages and frameworks or new PEPs. It is also great for discovering new and exciting projects.

Besides classic mailing lists, there is an official internet forum on the Discourse platform that is available at https://discuss.python.org/. It is a relatively new addition to the Python official discussion landscape and its purpose overlaps with many of the pre-existing mailing lists as it has dedicated categories for discussing new ideas, PEPs, and topics of core Python development. It has a lower entry barrier for those unfamiliar with the concept of mailing lists, and a much more modern user experience.

Unfortunately, not every new discussion happens on discuss.python.org, so if you want to know about everything that happens at the heart of Python development you will have to keep track of discussion both on the forum and mailing lists. Hopefully, these will eventually converge in a single place.

Besides the official message boards and mailing lists, there are a couple of open Python communities based on popular discussion and messaging platforms. The most notable ones are:

• PySlackers Slack workspace (pyslackers.com): A large community of Python enthusiasts using the Slack messaging platform to discuss anything Python-related
• Python Discord server (pythondiscord.com): Another open Python community but using Discord as their messaging platform
• /r/python subreddit (www.reddit.com/r/Python/): A single subreddit on the Reddit platform dedicated to Python

These three communities are open in the sense that you can freely join them as long as you are a user of their underlying platforms (which are, of course, free). Whatever you choose will probably be dictated by your preference for the specific messaging or discussion platform. The chances are high that you or one of your friends is already using one of these platforms.

The unquestionable advantage of such open communities is that they gather a very large number of members, and you can almost always find someone online that you can talk to. This provides the opportunity for ad hoc and loose discussions about various Python-related topics and allows you to seek quick help in case of simple programming problems.

The downside is that it is impossible to keep track of every discussion happening there. Fortunately, these communities often create systems of separate subchannels or tags that you can subscribe to if you want to be notified about the content of your specific interest. Also, these communities aren't officially endorsed and curated by the PSF. As a result, information found on Reddit or other online communities may sometimes be biased or inaccurate.

Other resources

Reading all the new PEPs, following mailing lists, and joining communities are the best ways to know what is happening at the moment. Unfortunately, doing this consistently requires a lot of time and effort as you will have to filter through huge amounts of information. Also, sources like mailing lists, message boards, and messaging platforms can be full of emotions because discussion is essentially a human interaction. And surprisingly, some tech discussions on contentious topics can be so heated that they are almost indistinguishable from social network drama.

If you are really busy or quickly get overloaded with social interactions between online strangers, there is another way. Instead of filtering online information all by yourself, you can turn to curated content like blogs, newsletters, and so-called "awesome lists."

Newsletters are especially good if you want to be up to date. These are some interesting newsletters that are worth subscribing to:

• Python Weekly (http://www.pythonweekly.com/) is a popular newsletter that delivers to its subscribers dozens of new, interesting Python packages and resources every week
• PyCoder's Weekly (https://pycoders.com) is another popular weekly newsletter with a digest of new packages and interesting articles

These newsletters will keep you informed about the most important Python announcements. They will also help you discover new blogs or especially interesting discussions from other discussion platforms, like Reddit or Hacker News. Keep in mind that the content of many general Python newsletters often overlaps, so probably there is no need to subscribe to all of them.

A completely different way to source information is via awesome lists. These are curated lists of links to valuable or important resources on specific topics maintained usually as Git repositories on GitHub. These lists are often very long and split into several categories.

These are examples of popular Python-related awesome lists curated by various GitHub users:

• awesome-python by vinta (https://github.com/vinta/awesome-python): This contains numerous references to interesting projects (mostly hosted on GitHub) and standard library modules divided into over 80 thematic categories. Categories range from basic programming concepts—like caching, authentication, and debugging—to whole engineering areas where Python is often used, like web development, data science, robotics, and penetration testing. The list of projects is supplemented with a collection of links to newsletters, podcasts, books, and tutorials.
• pycrumbs by kirang89 (https://github.com/kirang89/pycrumbs): This is focused on interesting and valuable articles. Articles are divided into over 100 categories dedicated to specific Python features, general programming techniques, and self-development topics.
• pythonidae by svaksha (https://github.com/svaksha/pythonidae): This is focused on specific fields of science and technology where Python is frequently used, like mathematics, biology, chemistry, web development, physics, image processing, and many more. It has a tree-like structure. The main page contains a list of over 20 main category pages. These categories contain more granular subcategories listing useful libraries and resources.

Awesome lists usually grow over time to enormous sizes. This means they are not a good resource for keeping yourself constantly updated. This is because they are simply snapshots of what the curators considered awesome at the time. Nevertheless, if you need to jump into a completely new area, let's say artificial intelligence, they serve as a good starting point for further research.

Summary

In this chapter, we've discussed the current status of Python and the process of change that is visible throughout the history of that language. We've learned why Python is changing and why it is important to follow that change.

Staying relevant is one of the biggest and most stressful challenges that professional programmers face regardless of the programming language they choose. Due to over 30 years of Python history and the ever-growing Python community, it isn't always clear how to efficiently stay up to date with the changes in the Python ecosystem. That's why we've looked into all the places where you can follow important discussions about Python's future.

What also changes together with the language are development tools that aim to ease and improve the software development processes. In the next chapter, we will continue with the topic of change and take a look at modern development environments. We will learn how to create repeatable and consistent runtime environments both for production and development use. We will also get familiar with various productivity tools provided by the Python community.